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Performance Assessment on a Platter — But will Teachers Partake? 



PALS (Performance Assessment in Learning Science) is an on-line 
(http://www.ctl.sri.com/pals/) standards-based, continually updated resource bank of 
science performance assessment tasks. PALS was founded on the premise that teachers 
will use well-developed performance assessments, especially if they are connected to 
standards. These could be the standards developed by national organizations (National 
Science Education Standards), individual states or even school districts. The point is that 
PALS was developed to fulfill a need that teachers, curriculum staff and assessment 
directors have been concerned about for years. 

The heart of this PALS system includes the performance tasks collected from numerous 
state and national sources. These are not isolated student problems but rather a complex 
combination of student directions and response forms, administration procedures, scoring 
rubrics, examples of student work, and technical quality statistics calculated from field- 
testing. On-line rater training packets have also been created for some tasks. 

Features of the Tasks Section of the PALS site include: 

1. Clear assessment targets that provide descriptions of specific achievement 
expectations. 

2. A focused purpose that clarifies the intended uses of the assessment results by 
teachers, curriculum developers, and policymakers. 

3. Use of an assessment method that matches performance goals. 



4. A representative sample of science performance tasks that teachers can use across 
grades to evaluate student knowledge. 

5. A way to produce evaluation results of maximum quality at minimum cost in time 
and effort. 

6. Accurate assessments free of bias. 

This paper is a case study of one urban district that had an opportunity to access this 
PALS site during the 1999-2000 school year. It details what worked and what did not 
and why. It also includes some recommendations from teachers about future 
improvements. 

The site of this study is a kindergarten through high school district in the mid-west of 
approximately 15,000 students located in 35 schools. Demographically, its percent of 
students living below the poverty level has increased 11% over the past four years to 52% 
while the State’s percentage increased only 4% for a total of 36%. The district's mobility 
rate is 34% or almost twice as high as the State’s average mobility rate of 18%. 

Some additional information about the district is necessary. The superintendent and local 
Board of Education realized several years ago that a laissez-faire approach to curriculum 
and instruction would yield laissez-faire achievement results. Therefore, while the State 
Board of Education generated a series of learning standards and benchmarks that crossed 
a span of grades, the district staff fine-tuned this effort by developing standards and 
benchmarks at each grade level through eighth grade and for each course at high school. 
Subsequently, staff are developing a “body of knowledge” to help explain and guide each 
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teacher in understanding what the standards and benchmarks really mean in standards 



based classrooms. The next step is to develop performance tasks as part of an assessment 
strategy to evaluate student progress. Given this curriculum approach, PALS would seem 
to be an excellent fit. 



In addition, this district has had a Project LINCOL’N (Living In the New Computer 

Oriented Learning ‘Nvironment) for the last five years. The Project’s purposes are to: 

• Promote science literacy by empowering differently abled learners to utilize a variety 
of technologies in an integrated, science-anchored curriculum 

• Restructure the year-long curriculum into thematic units anchored in science 

• Allow students to use the scientific method in inquiry-based experiences 

• Use mathematical and technological tools to research, explore and present newly 
constructed knowledge in authentic applications 

• Involve all students, including those with handicapping conditions, as interactive 
learners using multimedia technologies in cooperative groups facilitated by teachers 

• Enhance learning though parental involvement and extension of access to technology 
beyond the school day 

• Require students to apply knowledge and skills from all areas of the curriculum 
through culminating projects that focus on collaborative problem solving. 

A variety of technologies and multimedia are used by and with the students in a 

cooperative learning environment. Academic growth is documented through individual 

portfolios. 



One subsequent outcome of Project LINCOL’N has been the purchase and use of the 
FOSS science kits from kindergarten through fifth grade. This approach emphasizes the 
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use of a thematic approach in science and kits for instruction instead of forcing both the 
teachers and students to be bound by the use of a single textbook. 

Two years ago staff requested an evaluation of Project LINCOL’N to ascertain its long- 
term effects. Unfortunately, since this was an historical study, the recently developed 
PALS site was not accessible for this evaluation as an assessment resource. Therefore, 
the State test and a publisher’s multiple-choice test (mathematics and science sub-tests) 
were used over the objections of participating teachers. It was the program teachers’ 
belief that since their instructional approach emphasized a student’s ability to develop 
projects and solve complex problems, all multiple choice tests were not considered 
“authentic” enough to be valid measures. However, the results did show that while using 
two cohorts of students and two different science and mathematics tests over five years, 
that students who had gone through Project LINCOL’N surpassed their peers in the 
district and state as well as in national comparisons. 

The PALS site was first introduced to staff during a special training program funded via 
Goals 2000 funds during the summer of 1999. While originally Project LINCOL’N was 
located only in the Lincoln School fifth and sixth grade center, soon its philosophy had 
spread to other schools and other grade levels in the district. The necessary training for 
other district staff was best handled via a six-week summer training session. (A special 
one-week course for teams from other schools districts to undergo a mini version of this 
training has been funded by the State for several years.) 
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During this 1999 summer training a number of teachers — both elementary and high 
school— were exposed to PALS and given a chance to access tasks. Their immediate 
reaction was that the site gave them access to a variety and breadth of performance 
assessment tasks, scoring rubrics and other materials that they had never seen before. 
The philosophy of the site matched perfectly with their philosophy of instruction, 
especially since the development and use of rubrics had always been part of the summer 
training. 

Before the follow-up of these teachers during the school year is discussed, it should be 
noted that the use of PALS as a staff development tool during the district’s emerging 
standards training program was proposed. District administrators decided that if the 
district was really going to be standards based, all teachers would undergo a training 
program that would assist them in focussing on the standards and benchmarks in their 
classrooms. All kindergarten through third grade teachers had to be involved in this 
first year since a voluntary training program usually taps the same few hardy souls each 
year. The district’s standards coordinator was a former middle school science teacher 
and a PALS advocate so the addition of PALS to the four-day training program was a 
serious consideration. The teachers from several schools and their principals would 
attend two separate two-day sessions about a month apart. 



However, it was decided to make PALS a serious part of the 2000-2001 school year 
training when all the fourth through eighth grade teachers would be involved. A review 
of the PALS tasks did not provide many examples for teachers of students below fourth 
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grade. Instead, during the 1999-2000 school year, teachers in the younger grades were 
exposed to the potential of PALS. The training staff themselves and a number of science 
teachers throughout the district were involved in the review and reactions to PALS that 
are noted below. At the conclusion of the training next year and the following year 
(when all the high school teachers go through the district’s training process), a rich 
source of analyses should be available. 

In preparation for all these training sessions, each learning standard (including science) 
was placed on a matrix with grades from kindergarten through eighth grade and by course 
at high school. In the cells of these matrices are the assessments that measure student 
progress. So at a glance, one could see where there were holes either in instructional 
coverage of the standards or in their assessment. The plan is to encourage teachers to see 
how PALS tasks help close many assessment gaps. 

An additional note. Since PALS depends upon Internet access, it is critical to determine 
how easily teachers can find and use a computer with an Internet connection within their 
buildings. During this school year, the district’s last building will receive its fiber optic 
wiring so each classroom will have high speed Internet access. By the start of the 2000- 
01 school year, each teacher will have a computer available for his or her own use (wired 
to the Internet via a fiber optic connection). Four different levels of training are 
occurring this year so the teachers in each building will feel comfortable using the 
computer and “surfing the web” to find and access sites such as PALS. Stiggins, King, 
Fullan and others have long talked about the importance of teachers being assessment 
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literate but at the same time we have to provide the training and opportunities to make 
them computer literate if they are to partake of the advantages of sites such as PALS. 

The above paragraphs have set the context for looking at the effect of PALS during the 
first half of the 1999-2000 school year in one mid-west urban district. The rest of the 
paper will detail the results of interviews, observations and questionnaires that were 
completed by a number of district teachers. 

Regardless of grade level, all teachers are positive about the potential of PALS. They 
had not previously been exposed to a resource that gave them the entire picture — student 
directions, rubrics and student work samples along with the relevant performance task. 
However, many teachers stated that generally they neither had the time nor the interest in 
reviewing the “technical quality” of the tasks. But some obviously did because a few 
noted that there were tasks with missing information in this category. 

After the initial enthusiasm, there seemed to be a decline in the use of the PALS site by 
teachers. They said that they did not like to go through extensive searches. Teachers felt 
that they did not have the time or the patience to do the necessary searches. They did not 
want to loiter at the site; rather they wanted to quickly enter and leave. One 
recommendation to speed up the search would be to organize the PALS content by 
headings such as Magnets with the appropriate tasks grouped under it. 
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Other suggestions included quicker access. For example, if teachers are presenting a unit 
on “insects” between kindergarten and fourth grade, ideally they should be able to type 
that word in and then be directed immediately to any PALS tasks in which the word 
“insects” is used as part of the problem. This was especially relevant to these teachers’ 
needs because of the use of the FOSS curriculum which is centered on specific themes. 
Teachers also recommended that PALS tasks that can be directly linked to FOSS units 
should somehow be noted — especially when some tasks call for materials that are already 
part of the FOSS kits. A number of teachers stated that if the PALS tasks were pre- 
aligned by FOSS module, they would be more motivated to use the PALS site as a 
resource. 

Teachers liked the PALS site because it emulates the classic approach of gathering data, 
making observations, solving a problem and writing a conclusion. “Follow Those 
Tracks” is an example of a fourth grade life science task (contributed by the New York 
State Department of Education) that teachers liked because of the possibility of multiple 
right answers. 

Follow Those Tracks. Observe a picture containing patterns of animal 
tracks, infer what might have happened based upon the pattern of 
tracks, and write a story describing what might have happened. 

“Creative or alternative interpretations” of what happened to the animals whose tracks 

were found in the snow can be scored as correct. “Scorers should be open to these 

possibilities and not penalize student responses.” Teachers believed that such a problem 

correctly encourages students to consider alternative right solutions. 
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Several fourth grade teachers mentioned the example of Making Marbles as an excellent 

task that kept their students engaged. This performance task was contributed by the 

Council of Chief State School Officers (CCSSO/SCASS) 

Students study 3 marbles of different sizes. They make observations 
and measurements, organize their data, and design an experiment to 
answer questions. The accompanying scenario states: Super Toys is a 
company that manufactures toys. The company has been making only 
the middle-sized marbles you see on your plate. Marbles are made by 
pouring melted glass into molds. The company melts enough glass in 
each container to make 100 middle-sized marbles each time. How 
many small marbles can be made from one container of glass? How 
many large marbles can be made from one container of glass? Students 
are told they may use whatever equipment (from a variety of measuring 
equipment provided on their table) to do experiments to help answer 
the questions. This task is designed to take students approximately 45 
minutes to complete. 



Overall Task Content Area : Physical Science 

Specific Knowledge Areas : Properties of objects and materials 

Performance Expectations for the students include: 

• conducting investigations 

• using equipment 

• gathering, organizing, and representing data 

• formulating conclusions from investigational data 

• applying scientific principles to develop explanations and solve new problems 




Teachers found this task by going to the section of PALS where they could access 
different sets of content standards, including Illinois. They first chose which Illinois 
Learning Standard (ILS) in Science that they wished to use and the appropriate grade 
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range (k-4). In this case they chose: State Goal 11 (from ILS): Understand the processes 
of scientific inquiry and technological design, to investigate questions, conduct 
experiments and solve problems . The first benchmark under this goal states: Know and 
apply the concepts, principles and processes of scientific inquiry (1 l.A). Under this the 
teacher can check which one of these statements that matches her plans. The two 
learning outcomes that were chosen from this following list are in italics. 

• Describe an observed event (1 l.A. la) 

• Develop questions on scientific topics (1 l.A. lb) 

• Collect data for investigations using measuring instruments and technologies 

(ll.A.lc) 

• Record and store data using available technologies ( ll.A.ld ) 

• Arrange data into logical patterns and describe the patterns ( ILA.le) 

• Compare observations of individual and group results (1 l.A. If) 

• Formulate questions on a specific science topic and choose the steps needed to 
answer the questions (1 l.A.2a) 

• Collect data for investigations using scientific process skills including observing, 
estimating and measuring (ll.A.2b) 

• Construct charts and visualizations to display data (1 l.A.2c) 

• Use data to produce reasonable explanations (1 l.A.2d) 

• Report and display the results of individual and group investigations (1 l.A.2e) 
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The teachers noted which ones they wished to conduct further study on and selected the 
“Show Assessment Chart” button. That action produced a chart with the two above 
checked statements in the left hand column and a matrix going to the right listing 27 
different possible performance tasks that could be used, including eleven that could be 
used to measure both student learning outcomes. That matrix also appears as a list that 
the teacher can then check to access all the particular information about this task. In this 
case the task was the Making Marbles performance task. 



Teachers liked the opportunity to search by Illinois standard, such as in the example in 
the previous paragraphs. They also were positive about the idea that tasks are cross- 
referenced when they fit multiple standards. However, those teachers who are familiar 
with the Illinois Learning Standards would like the ability to search by standard from the 
beginning page. They recommended a pull down window that would appear on the 
PALS opening page so they would not have to weave their way through the site. 



Fullan has stated that “...the more powerful technology becomes, the more indispensable 
good teachers are. Technology generates a glut of information, but it has no particular 
pedagogical wisdom.” One of our conclusions from the PALS experience this year is 
that district users have to avoid the temptation to use a passive presentation approach. To 
those who have long sought this kind of resource, it seems quite clear how the site can 
become an integral part of classroom assessment for teachers. However, our experience 
has shown us that: 
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• We have to help teachers focus on making the connections between PALS and our 
own district standards, assessment strategy, reporting assessment results, curriculum 
and school improvement planning. 

• We have to continue to ensure that the teachers feel comfortable using their 
computers to access and use sites such as PALS. There is an apparent contradiction 
between those staff who want to quickly move into the site grab the needed resource 
and retreat and those staff with training responsibilities. The latter believe that PALS 
is a great site since it “...provided examples of good and bad rubrics as tools to guide 
teachers in creating their own rubrics... that teachers can gain a variety of skills while 
exploring PALS’ databases, including illustrative examples of how to use a rubric to 
improve their own consistency in scoring performance assessments.” 

• We will directly link our new district assessment page to PALS along with 
recommendations about how our staff can best use the site. 

• We will develop a strategy this summer to provide follow-up training in their schools 
to staff who went through the first year of district standards training and we will 
incorporate PALS as an integral component of the training for the district’s grade four 
through eighth grade teachers next year. 

We recommend that PALS developers: 

• Develop the capacity to do a word search to identify tasks with grade level 
combinations. Perhaps an increased number of teachers can be enticed to explore 
PALS more thoroughly and more often with a few quick successes. 




Display a button on the first page than enables one to search the tasks by national or 
state standard. Again, such a short cut may encourage more participation. 
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• Some of the original performance tasks included lessons (e.g., how airplanes fly for 
middle school students) that also could be an excellent resource to encourage more 
teacher participation. Can these be found and included? 

• Investigate the possibility of tying the performance tasks to curriculum such as FOSS 
just as they are tied to some state as well as national standards. 

• Continue to lobby for funds to develop a site for the Performance Assessment for 
Learning Mathematics. 
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